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Abstract 

The newly proposed l\ norm constraint zero-point attraction Least Mean Square 
algorithm (ZA-LMS) demonstrates excellent performance on exact sparse system iden- 
tification. However, ZA-LMS has less advantage against standard LMS when the system 
is near sparse. Thus, in this paper, firstly the near sparse system modeling by General- 
ized Gaussian Distribution is recommended, where the sparsity is defined accordingly. 
Secondly, two modifications to the ZA-LMS algorithm have been made. The l\ norm 
penalty is replaced by a partial l\ norm in the cost function, enhancing robustness with- 
out increasing the computational complexity Moreover, the zero-point attraction item 
is weighted by the magnitude of estimation error which adjusts the zero-point attraction 
force dynamically. By combining the two improvements, Dynamic Windowing ZA-LMS 
(DWZA-LMS) algorithm is further proposed, which shows better performance on near 
sparse system identification. In addition, the mean square performance of DWZA-LMS 
algorithm is analyzed. Finally, computer simulations demonstrate the effectiveness of 
the proposed algorithm and verify the result of theoretical analysis. 

Keywords: LMS, Sparse system identification, Zero-point attraction, ZA-LMS, 
Generalized Gaussian distribution 



1 Introduction 

A sparse system is denned when impulse response contains only a small fraction of large 
coefficients compared to its ambient dimension. Sparse systems widely exist in many appli- 
cations, such as Digital TV transmission channel pQ and the echo path [2]. Generally, they 
can be further classified into two categories: exact sparse system (ESS) and near sparse 
system (NSS). If most coefficients of the impulse response are exactly zero, it is defined as 
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an exact sparse system (Fig. [TJ a); Instead, if most of the coefficients are close (not equal) 
to zero, it is a near sparse system (Fig. [TJ b and c). Otherwise, a system is non-sparse if 
its most taps have large values (Fig. [TJ d). For the simplicity of theoretical analysis, sparse 
systems are usually simplified into exact sparse. However, in real applications most systems 
are near sparse due to the ineradicable white noise. Therefore, it is necessary to investigate 
on near sparse system modeling and identification. 

Among many adaptive filtering algorithms for system identification, Least Mean Square 
(LMS) algorithm [3], which was proposed by Widrow and Hoff in 60s of the past century, 
is the most attractive one for its simplicity, robustness and low computation cost. How- 
ever, without utilizing the sparse characteristic, it shows no advantage on sparse system 
identification. In the past few decades, some modified LMS algorithms for sparse systems 
are proposed. M-Max Normalized LMS (MMax-NLMS) @] and Sequential Partial Update 
LMS (S-LMS) [5] reduces the computational complexity and steady-state misalignment by 
partially updating the filter coefficients. Proportionate LMS (PLMS) [2] and its improved 
ones such as IPNLMS [B] and IIPNLMS [7] accelerate the convergence rate by updating 
each coefficient iteratively with different step size proportional to the magnitude of filter 
coefficient. Stochastic Tap-Normalized LMS (ST-NLMS) [8l[9] improves the performance 
on specific sparse system identification where large coefficients appear in clusters. It locates 
and tracks the non-zero coefficients by adjusting the filter length dynamically. However, its 
convergence performance largely depends on the span of clusters. If the span is too long 
or the system has multiple clusters, it shows no advantage compared with standard LMS 
algorithm. 

More recently, inspired by the research of CS reconstruction problem [1U|.I1 lj . a class 
of novel adaptive algorithms for sparse system identification have emerged based on the l p 
(0 < p < 1) norm constraint |12H14j . Especially, Zero-point Attraction LMS (ZA-LMS) 
algorithm [12] significantly improves the performance on exact sparse system identification 
by introducing a l\ norm constraint on the cost function of standard LMS, which exerts 
the same zero-point attraction force on all coefficients. However, for near sparse systems 
identification, the zero-point attractor can be a double-edged sword. Though it increases 
the convergence rate because by the l\ norm constraint, it also produces larger steady-state 
misalignment as it forces all coefficients to exact zero. Thus, it possesses less advantage 
against standard LMS algorithm when the system is near sparse. In this paper, firstly Gen- 
eralized Gaussian Distribution (GGD) [15] is introduced to model the near sparse system. 
Then two improvements on the ZA-LMS algorithm is proposed. Above all, by adding a 
window on the l\ norm constraint, the steady-state misalignment is reduced without in- 
creasing the computational complexity. Furthermore, the zero-point attractor is weighted 
to adjust the zero-point attraction by utilizing the estimation error. By combining the 
two improvements, the Dynamic Windowing ZA-LMS (DWZA-LMS) algorithm is proposed 
which shows improved performance on the near sparse system identification. 

The rest of the paper is organized as follows: In Section II, ZA-LMS algorithm based on 
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(c) Generalized Sparse System generated by GGD, where P=0.1, o~=l 
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Figure 1: Sparse and non-sparse systems, (a) is a exact sparse system, (b) and (c) are 
near sparse system generated by Generalized Gaussian Distribution with j3 = 0.05 and 0.1 
respectively, (d) is a non-sparse system generated by Gaussian distribution. 

l\ norm constraint is reviewed, and the near sparse system is modeled. The new algorithm 
is proposed in Section III. In Section IV, the mean square convergence performance of 
DWZA-LMS is analyzed. The performances of the new algorithm and other improved LMS 
algorithms for sparse system identification are compared by simulation in Section V, where 
the effectiveness of our analysis is verified as well. Finally, Section VI concludes the paper. 

2 Preliminary 

2.1 Review of ZA-LMS algorithm 

Let d(n) be a sample of the desired output signal 

d(n) = h T x(n) + v(n), (1) 

where h = [ho,h\,--- ,/il_i] T is the unknown system with memory length L, x(n) = 
[x(n),x(n — 1), • • • ,x(n — L + 1)] T denotes the input vector, and v(n) is the observation 
noise assumed to be independent of x(n). The estimation error e(n) between desired and 
output signal is defined as 

e(n) = d(n) — w T (n)x(n). (2) 

where w(n) are the filter coefficients and w(n) = [wo(n),wi(n), ■ ■ ■ , wx,_i(n)] T . Thus the 
cost function of ZA-LMS is 

1 2 
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where ||w(n)||i denotes the l± norm of the filter coefficients. Parameter rj is the factor 
balancing the new penalty and the estimation error. By minimizing ([3]), the ZA-LMS 
algorithm updates its coefficients by 

w(n + 1) = w(n) + fie(n)x(n) — psgn [w(n)] , (4) 

where /i is the step size, p = 77/2 is the zero-point attraction controller, and sgn[-] is the 
component- wise sign function. 

By observing @, the recursion of filter coefficients for sparse system can be summarized 

as 

Wncw = W prcv + /U/gc(w) + p/zA(w), (5) 

where /gc( w ) = e(n)x denotes the gradient correction function, and /za(w) = — sgn(w) 
stands for zero-point attractor which is caused by the l\ norm penalty. For each iteration, 
the zero-point attractor forces the filter taps to decrease a little when it is positive, or 
otherwise to increase a little when it is negative. 



2.2 Near Sparse System Modeling 

Exact sparse system is appropriate for theoretical analysis, however, most physical sys- 
tems are near sparse with widely existing white noise in real life, thus their modeling is of 
more significant importance. Generalized Gaussian Distribution (GGD) is one of the most 
prominent and widely used sparse distributions. For example, in multimedia communica- 
tions such as image and speech coding, GGD is usually found to best fit the coefficients of 
the discrete sine and cosine transforms, the Walsh-Hadamard transform, and the wavelet 
transform |16j . In ultra wide bandwidth (UWB) systems, it has recently been found to 
fit the multiuser interference better [17J. These findings lead to applications of GGD in 
video coding, speech recognition, blind signal separation and UWB receiver design |18H20] , 
Therefore GGD is utilized to model the near sparse system in this study. It is a class of 
symmetry distribution with the Gaussian and Laplacian distribution as the special cases, 
with delta and uniformity distribution as limit. The probability density function of GGD 
is 

m = 2 \T(i/(3) ew H x " H/A)/ 1 ' (6) 



where A = °"g\/ 1 T( x ) denotes the Gamma function, fj, and a 2 „ are called the mean and 
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variance of GGD, respectively. Besides, f3 determines the decay rate of the density function 
and can be used to denote the sparsity of the system (see as Fig. [TJ b, c; please notice that 
the system sparsity decreases as [3 increases). Especially, GGD is Gaussian Distribution 
when (3 = 2, and it turns into Laplacian Distribution when f3 = 1. By integrating f(x), the 
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Figure 2: Comparison between ZA-LMS and LMS on Exact Sparse System and Near Sparse 
System. 



distribution function is 
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where @(s,x) = Jjf t s ~ l e~ t dt is called the lower incomplete function. 



(7) 



3 Improved ZA-LMS Algorithm 

The ZA-LMS algorithm improves the performance by exerting an l\ norm penalty forcing 
small coefficients to zero iteratively. It is very effective for the exact sparse system iden- 
tification. However, for the near sparse system with small white noise on all coefficients, 
most of the small coefficients are forced to zero. Thus ZA-LMS usually degrades with large 
steady-state misalignment, showing no improvement compared with standard LMS. The 
situation is more clearly illustrated in Fig. [2j where the exact sparse sparse system is gen- 
erated with 8 nonzero large coefficients with tap length of 100, and the near sparse system 
is the same with the former except that a power of 1 x 10~ 4 white noise is added on all 
coefficients. The step sizes for all algorithms are set as fi = 0.01. The optimal parameter p 
is derived theoretically from (36) of |21] by minimizing the steady-state MSD. 

From Fig. [2j it can be seen that by choosing the theoretically optimal parameter p the 
performance of ZA-LMS is much better than standard LMS for exact sparse system. How- 
ever, the performance of ZA-LMS severely degrades on near sparse system identification. 
As stated above, the main reason is that the strong zero-point attraction forces near zero 
coefficients to zero that results in the large steady-state misalignment. Though empirically 
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Figure 3: The zero-point attractor for sparse system, (a)ZA-LMS, (b)WZA-LMS. 

decreasing p may alleviate this problem, in that case, ZA-LMS will degrade to standard 
LMS showing no improvement, as shown in Fig. [2j Thus to improve the performance of 
near sparse system identification, a new ZA-LMS based algorithm is put forward, the main 
modifications are as follows. 



As the attracting range of ZA-LMS algorithm reaches infinity, all coefficients of the sparse 
system are attracted to zero point. However, the identical attraction on both large and small 
ones will lead to increase the computational complexity and steady-state misalignment. 
Thus, the first improvement lies in the constraint of zero-point attracting range. As shown 
in Fig. 3, the new zero-point attraction, which attracts the coefficients only in a certain 
range, is proposed by adding a window on the original zero-point attracting. The new 
recursion of coefficients of the proposed Windowing ZA-LMS (WZA-LMS) is 



where a and b are both positive constant which denotes the lower and upper threshold of 
the attraction range, respectively. 

From Fig. 3, it can be concluded that ZA-LMS is the special case of WZA-LMS when a 
reaches and b approaches infinity, respectively. Besides, by investigating (jlj), © and ([9]), 
it can be seen that the computational complexity of the two algorithms is approximately 
the same. And by adopting the new zero-point attractor and properly setting the threshold, 
the coefficients, whether too small or too large, will not be attracted any more. Thus, the 
steady-state misalignment is significant reduced especially for near sparse system. 



3.1 Windowing ZA-LMS 



w(n + 1) = w(n) + /ie(n)x(n) — psgn^, [w(n)] 



(8) 



where sgn w [-] is the component- wise partial sign function, defined as 




a < \t\ < b; 
elsewhere. 



(9) 
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3.2 Dynamic ZA-LMS 

As mentioned above, the sparse constraint should be relaxed in order to reduce the steady- 
state misalignment when the updating procedure reaches the steady-state. Inspired by 
the idea of variable step size methods of standard LMS algorithm [23], the magnitude of 
estimation error, which denotes the depth of convergence, is introduced here to adjust the 
force of zero-point attraction dynamically. That is, 

/dzaW =/ W zA(*)|e(n)|. (10) 

At the beginning of iterations, large estimation error increases zero-point attraction force 
which also accelerates the convergence. When the algorithm is approaching the steady-state, 
the error decreases to a minor value accordingly. Thus the influence of zero-point attraction 
force on small coefficients is reduced that produce smaller steady-state misalignment. By 
implementation of this improvement on ZA-LMS algorithm, the algorithm is named as 
Dynamic ZA-LMS (DZA-LMS). 

3.3 Dynamic Windowing ZA-LMS 

Finally, by combining the two improvements, the final Dynamic Windowing ZA-LMS (DWZA- 
LMS) algorithm can be drew. The new recursion of filter coefficients is as follows, 

Wi{n + 1) = Wi{n) + fie(n)x(n — i) — p\e(n)\sgn w [wi(n)] V0 < i < L. (11) 

In addition, the new method can also improve the performance of ZA-NLMS, which is 
known for its robustness. The recursion of DWZA-NLMS is 

Wi(n + 1) = Wi(n) H ■ — ] {fie(n)x(n - i) - p |e(n)| sgn^ [wi(n)}} V0 < i < L. 

e + x 1 (n)x(n) 

(12) 

where e > is the regularization parameter. 

4 Analysis of the proposed Algorithm 

The mean square convergence analysis of DWZA-LMS algorithm is carried out in this 
section. The analysis is based on the following assumptions. 

1. The input signal x(n) is i.i.d zero-mean Gaussian. The observation noise v(n) is 
zero-mean white. The tap-input vectors x(n) and the desired response d(n) follow 
the common independence assumption [22], which are generally used for performance 
analysis of LMS algorithm. 

2. The unknown near sparse filter tap follows GGD. As stated in Section II, this assump- 
tion is made because GGD is the suitable sparse distribution for near sparse system 
modeling. 
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3. The steady state adaptive filter tap Wi(n) follows the same distribution with hi (1 < 
i < L). This is a reasonable assumption in that the error between the coefficients 
of the identified and the unknown real systems are very small when the algorithm 
converges. 

Under these assumptions, the mean square convergence condition and the steady-state 
MSE of DWZA-LMS algorithm are derived. Also, the choice of parameters is discussed in 
the end of this section. 

First of all, the misalignment vector is defined as 

A(n) = w(n) - h, (13) 
and auto-covariance matrix of A(n) as 

K(n) = E { A(?i)A T (n)} . (14) 

4.1 Mean Square Convergence of Misalignment 

Combining CQ, ([2]), (fTTI) and (fT3j) . one derives 

A(n + 1) = A(?i)A(n) + /_iv(n)x(n) — pm(n), (15) 

where A(n) = I — /ix(n)x T (n), m(n) = [mo(n), mi(n), • • • ,mi_i(n)], and 

rrii{n) = \e{n)\sgn w [wi{n)} VO < i < L. (16) 

By utilizing the independence assumption [3], and substituting (fT5j) into (fl~4"|) yields 

K(n + 1) = E{A(n)A(n)A T (n)A T (n)} + /iV 2 CT 2 I L 

-pE{m(n)A T (n)A T (n)} + p 2 E{m(n)m T (n)} 
-pE{A(n)A(n)m T (n)}, (17) 

where 1^ is an L x L unit matrix, cr 2 and <r 2 denote the power of input signal and observation 
noise, respectively. By utilizing the property that the fourth-order moment of a Gaussian 
variable is three times the variance square, one obtains 

E{A(n)A(n)A T (n)A T (n)} = (1 - 2/zcr 2 + 2// 2 ^)K(n) + fj, 2 a*D(n)I L , (18) 

where D(n) = tr [K(n)]. With ([13]), one has 

E{A(n)A(n)m T (n)} = (E{m(n) A T (n)A T (ra)}} T = (1 - /xcj 2 )E{A(n)m T (n)}. (19) 

Combining (fTTI) . (fT8l) and (fT9l) . one derives 

K(n + 1) = (1 - + 2/i 2 <r*)K(n) + ^ 2 cr^(n)I L 

+A 2 a 2 I x - p(l - ^ 2 .) {E{A(n)m T (n)}} T 

-p(l - ^cr 2 )E{A(n)m T (n)} + p 2 E{m(ra)m T (ra)}. (20) 
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By taking trace on both sides of (|20p . it can be concluded that the adaptive filter is stable 
if and only if 

< 1 - 2fxal + (L + 2)p?a A x < 1, (21) 

which is simplified to 

< ^ < (T MO\ 2 - ( 22 ) 

This implies that the proposed DWZA-LMS algorithm has the same stability condition 
for the mean square convergence as the ZA-LMS and standard LMS algorithm [3]. 

4.2 Steady-state Mean Square Error 

In this subsection, the steady-state Mean Square Error (MSE) of DWZA-LMS algorithm is 
analyzed. By definition, MSE is 

MSE = E{e 2 (oo)} = a 2 + tr{R(oo)K(oo)}, (23) 

where R(oo) = ct 2 Il, then ([23]) can be rewritten as 

MSE = <r 2 + o*D(oo). (24) 

Thus, our work is to estimate D(oo). In (|20|) . let n approach infinity, by observing the ith 
(0 < i < L) element iQ(co) of the matrix K(oo), one obtains 

Ki(oo) = (1- 2[ial + 2fi 2 ai)Ki(oo) + Li 2 a*D(oo) 
+H 2 a 2 v a 2 x - 2p(l - ^E^m^oc)} 

+p 2 E{m 2 (oo)} V0 < i < L, (25) 



With reference to (jl6j) . it is obvious that 

T^r 2/ n / MSE a < \ Wi (oo)\ < b; 

E{m?(oo)} = < (26) 
I L) elsewhere. 

To derive E{Aj(oo)mj(oo)}, by multiplying m T (?i) on the right of each item of (|15|) and 
taking the expectation value on both sides as well as letting n approach infinity it yields 

E{A(oo)m T (oo)} = ^E{m(oo)m T (oo)}. (27) 

Thus, when |u;j(oo)| < a or |u>j(oo)| > b (0 < i < L), it has 

_ fia 2 x D(co) n 2 a 2 x a 2 v 
2(1 -poi) 2iiai\\ - iiai) 



when a < |u;j(oo)| < b (0 < i < L), it has 



#i(oo) 



2/^2(1-^ 



/x 2 ^L>(oo) + /i 2 a 2 a 2 + ( ^ - 1 ) ^MSE 



(29) 
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According to Assumption (3), and combining ([7|), we have 



V{a< \wi(n)\ < b} 
2[F(b)-F(a)] 



e 







/r(V/3), 



(30) 



where Pa denotes the probability that the coefficients of adaptive filter will be attracted. 
On the other hand, the probability that they will not be attracted is Pna = 1 — Pa- By 
combining (I28p and (|29H and summing up all the diagonal items of matrix K(n), it yields 



L 



D(oo) 



1 MSE 



pa 2 [2-pa 2 (L + 2)] 
Combining (|24p and (|31|) , finally one has 

iial [2 - pa 2 (L + 2)] + Lp 2 a 2 x a 2 



MSE 



2p- p 2 a 2 {L + 2) - p 2 P A L 



/id 2 



1 



(31) 



(32) 



If p = 0, equation (I32p is the same with MSE of standard LMS algorithm [3], 

(2 - 2/^) CT 2 



MSElms 



2-pa 2 (L + 2) 



(33) 



4.3 Parameter Analysis 



The performance of the proposed algorithm is largely affected by the balancing parameter 
p and the thresholds a and b. 

According to © and dU, it can be seen that the parameter p determines the importance 
of the l\ norm and the intensity of zero-point attraction. In a certain range, a larger p, 
which indicates stronger attraction intensity, will improve the convergence performance by 
forcing small coefficients toward zero with fewer iterations. However, according to ()32|) . a 
larger p also results in a larger steady-state misalignment. So the parameter p can bal- 
ance the tradeoff between adaptation speed and quality. Moreover, the optimal parameter 
p empirically satisfies p <C <C 1. By analyzing steady-state MSE in (f32j) under such 
circumstance, it can be seen that 



P 2 PaL 



\pai 



1 



« \2p-p 2 a 2 x {L + 2)\. 



(34) 



According to (|34p . the influence of the last term in the denominator of (|32p can be ignored, 
which means that the steady-state MSE of the proposed algorithm is approximately the 
same with standard LMS for near sparse systems identification. The same conclusion can 
also be drawn from (jlip intuitively: when the adaptation reaches steady-state, the small 
e(n) renders the value of /?|e(n)| trivial compared to p, letting the relaxation of zero-point 
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attraction constraint. On the other hand, with large e(n) in the beginning and the process 
of adaptation which indicates larger zero-point attraction force, the zero-point attractor 
adjusts the small taps more effectively than ZA-LMS, forcing them to zero with fewer 
iterations, which accelerate the convergence rate significantly. 

The thresholds a and b determine the zero-point attraction range together. The param- 
eter a is set to avoid forcing all small coefficients to exact zero, it is suggested to be set as 
the mean amplitude of those near zero coefficients of the real system. Specifically, for exact 
sparse systems, as most coefficients of the unknown system are exactly zero except some 
large ones, accordingly a = is set to force most small coefficients to exact zero. For exact 
sparse systems contaminated by small Gaussian white noise, a should be set as the standard 
deviation of the noise. For near sparse systems generated by GGD, as the mean amplitude 
of the small coefficient is hard to derive, we empirically choose a for the proposed algorithm. 
As a small sparsity indicator (3 in GGD usually means smaller mean amplitude of the small 
coefficient, we choose smaller a when (3 is smaller. According to the simulations, a is chosen 
in the range 1 x 10~ 3 to 1 x 10~ 2 for GGD with /3 varying from 0.05 to 0.5. The parameter 
b is chosen to reduce the unnecessary attraction of large coefficients in ZA-LMS, therefore, 
empirically any constant b, which is much larger than the deviation of small coefficients 
and much smaller than infinity, should be appropriate. Various simulations demonstrate 
that the parameter b can be set as a constant around 1 for most near sparse systems. This 
choice of b is quite standard for most applications. 

5 Simulations 

In this section, first we demonstrate the convergence performance of our proposed algorithm 
on two near sparse systems and a exact sparse system in Experiment 1-4, respectively. 
Second, Experiment 5-7 are designed to verify the derivation and discussion in Section IV. 
Besides the proposed algorithm, Standard NLMS, ZA-LMS, IPNLMS [6] and IIPNLMS [7] 
are also simulated for comparison. To be noticed, the normalized variants of ZA-LMS and 
the proposed algorithm are adopted to guarantee a fair comparison in all experiments except 
the fourth, where DWZA-LMS is simulated to verify the theoretical analysis result. 

The first experiment is to test the convergence and tracking performance of the proposed 
algorithm on near sparse system driven by Gaussian white signal and correlated input, 
respectively. The unknown system is generated by GGD which has been shown in Fig. [1] b 
with filter length L = 100, it is initialized randomly with f3 = 0.05 and o 2 g = 1. For the 
white and correlated input, the system is regenerated following the same distribution after 
1700 and 4500 iterations, respectively. For the white input, the signal x(n) is generated by 
white Gaussian noise with power a 2 = 1. For the correlated input, the signal is generated 
by white Gaussian noise y(n) driving a first-order Auto-Regressive (AR) filter, x(n) = 
0.8x(n — 1) + y(n), and x[n) is normalized. Besides, the power of observation noise is 
a 2 = 1 x 10 -4 for both input. The five algorithms are simulated 100 times respectively with 
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Table 1: Comparison of computational complexity of IPNLMS, IIPNLMS and DWZA- 
NLMS 



Algorithms 




Multiplies 




Comparisons 


Convolution 


Tap update 


Total 


IPNLMS 


L 


3L 


4L 





IIPNLMS 


L 


3L 


4L 


4L 


ZA-NLMS 


L 


2L 


3L 





DWZA-NLMS 


L 


(l + Pi)L 


{2 + Pi)L 


2L 



Where Pi £ [0, 1] denotes the fraction of coefficients in the attracting range of DWZA- 
NLMS. 



parameter jjl = 1 in both cases. The other parameters are as follows 

• IPNLMS and IIPNLMS with white input: a P = -0.5, p = 0.4, a P1 = -0.5, a P2 = 0.5, 
r = 0.1; 

• IPNLMS and IIPNLMS with correlated input: a P = -0.5, p = 0.2, a P1 = -0.5, 
a P2 = 0.5, T = 0.1; 

• ZA-NLMS and DWZA-NLMS with white input: p ZA = 3 x 10~ 4 , powzA = 6 x 10~ 2 , 
a = 1 x 10" 3 , b = 0.8. 

• ZA-NLMS and DWZA-NLMS with correlated input: p Z A = 3 x 10" 4 , pdwza = 
3 x 10^ 2 , a = 1 x 10~ 3 , b = 0.8. 

All the parameters are particularly selected to keep their steady-state error in the same 
level. The MSD of these algorithms for both white and correlated input are shown in 
Fig. 4(a) and Fig. |4(b)[ respectively. The simulation results show that all algorithms con- 



verge more slowly in the color input driven scenario than in the white noise driven case. 
However, the ranks or their relative performances are similar and the proposed algorithm 
reaches the steady state first with both white and correlated input. On the other hand, 
the performance of ZA-NLMS degenerate to standard NLMS as the system is near sparse. 
Furthermore, the computational complexity of the proposed algorithm is also smaller com- 
pared with improved PNLMS algorithms (Table [1]). Besides, when the system is changed 
abruptly, the proposed algorithm also reaches the steady-state first in both cases. 

The second experiment is to demonstrate the proposed algorithm on near-sparse systems 
other than GGD. The near-sparse system with 100 taps is generated in the following manner. 
First, 8 large coefficients following Gaussian distribution M(0, 1) are generated, where all 
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(b) Correlated Input 



Figure 4: Comparisons of convergence rate and tracking ability of five different algorithms 
for sparse system identification driven by (a) white input and (b) correlated input, respec- 
tively. 



their tap positions follow Uniform distribution. Second, white Gaussian noise with variance 
o\ = 1 x is added to all taps, enforcing the system to be near-sparse. The signal x(n) 
is generated by white Gaussian noise with power a\ = 1. Five algorithms, the same as in 
Experiment 1, are simulated 100 times respectively with parameter fj, = 1. The parameters 
are set to the same values as in the white input case of Experiment 1 except a = 1 x 1CP 2 
for the proposed algorithm. From Fig. [5j we can conclude that the proposed algorithm 



13 




I I I I I i_v ^ *— * v 1 

200 400 600 800 1000 1200 1400 
Iterations 



Figure 5: Comparisons of convergence rate on NLMS, ZA-NLMS, IPNLMS, IIPNLMS, and 
the proposed algorithm on near sparse system. 

reaches the steady-state first in such near sparse system. 

The third experiment demonstrates the effectiveness of the proposed modification on 
ZA-LMS algorithm on near sparse system. The system and the signal are generated in the 
same way as in Experiment 2. The proposed DZA-NLMS, DWZA-NLMS are compared 
with ZA-NLMS for the system with 100 simulations. The step length for all algorithms are 
set as p, = 1. We particularly choose parameters pdza = 0.05 and pzA = 6.5 x 10~ 4 for 
DZA-NLMS and ZA-NLMS to ensure their steady-state mean square error in the same level. 
We set pdwza = 0.05 for DWZA-NLMS algorithm for a fair comparison with DZA-NLMS. 
The parameter a and b in DWZA-NLMS are chosen as a = 0.01 and b = 0.8, respectively. 

From Fig. [6l we can see that with the dynamic zero-point attractor DZA-NLMS con- 
vergences faster than ZA-NLMS. By adding another window constraint on the zero-point 
attractor, The DWZA-NLMS not only preserves the property of fast convergence of DZA- 
NLMS, but also shows smaller steady-state mean square error than both ZA-NLMS and 
DZA-NLMS. 

The fourth experiment shows that the proposed improvement is still effective on the 
exact sparse system identification. The unknown system is shown in Fig. [TJ a, where 
filter length L = 100. Besides, 8 large coefficients is uniformly distributed and generated 
by Gaussian distribution M(0, 1), and all other tap coefficients are exactly zero. The input 
signal is generated by white Gaussian noise with power a\ = 1, and the power of observation 
noise is cr^ = 1 x 10~ 4 . The proposed algorithm is compared with ZA-NLMS and standard 
NLMS, where each algorithm is simulated 100 times with 2000 iterations. The step size p 
is set to 0.65 for NLMS, and p = 1 for both the ZA-NLMS and the proposed algortihms. 
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Figure 6: Comparisons of convergence speed on ZA-NLMS, DZA-NLMS and DWZA-NLMS. 
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Figure 7: Comparisons of convergence rate on NLMS, ZA-NLMS, and the proposed algo- 
rithm on exact sparse system. 



The parameter pzA is set to 6 x 10~ 4 and /?dwza = 6 x 10~ 2 . All the parameters are chosen 
to make sure that the steady-state error is the same for comparison. According to Fig. [7J it 
can be seen that the convergence performance is also improved compared with ZA-NLMS 
via the proposed method on exact sparse system, thus the proposed improvement on the 
algorithms are robust. 

The fifth experiment is to test the sensitivity to sparsity of the proposed algorithm. 
All conditions are the same with the first experiment except the sparsity. The parameter 
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Figure 8: Comparisons of convergence rate of proposed algorithm for different sparsity. 

(3 is selected as 0.05,0.1, and 0.15, respectively. Besides, we also compared our algorithm 
when the system is non-sparse which is generated by Gaussian distribution. For each f3, 
both the proposed algorithm and NLMS are simulated 50 times with 1700 iterations. The 
step size p is 1 for both algorithms, and p = 4 x 10~ 2 , a = 1 x 10~ 2 , b = 0.8 for the 
proposed algorithm. The simulated MSD curves are shown in Fig. The steady-state 
MSD remains approximately the same for the proposed algorithm with varying parameter 
(3 which denotes the sparsity of systems, meanwhile the convergence rate decreases as the 
sparsity decreases. For the non-sparse case, our algorithm degenerates and shows similar 
behavior with standard NLMS. However, for each /3 the proposed algorithm is never slower 
than standard NLMS. It should be noticed that NLMS is independent on the system sparsity 
and behaves similar when /3 varies. 

The sixth experiment is to test the steady-state MSE with different parameters. The 
coefficients of unknown system follows GGD with filter length L = 100, the sparsity and 
variance are chosen as cr 2 = 1 and j3 = 0.1, respectively. The input is generated by white 
Gaussian noise with normalized power a\ = 1, and the power of observation noise is 1 x 10 -4 . 
Under such circumstance, the steady-state MSD is tested. Here a = 1 x 10~ 2 and b = 0.8 are 
set for each simulation. The step size p is varied from to 1.1 x 10 -3 for given p = 2 x 10 -4 . 
And p is changed from to 1 x 10~ 3 for given p = 1 x 10~ 2 . Fig. [U]and Fig. [TU]show that 
the analytical results accord with the simulated ones of different parameters for variable 
values. Specifically, in Fig. [9j the steady-state MSE goes up as the step size increases, 
whose trend is the same with standard LMS. Fig. [TU1 shows that the analytical steady-state 
MSE matches with simulated one as the parameter p gets larger, which verifies the result 
in Section V. 



16 



-Q— Simulated MSE 
—\ — Analytical MSE 




x 10" 



Figure 9: The steady-state MSD of the proposed algorithm with different step size 
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Figure 10: The steady-state MSD of the proposed algorithm algorithm with different 
parameters p 



The seventh experiment is designed to test the behavior of the proposed algorithm with 
respect to different parameters of a and b. All conditions of the proposed algorithm are the 
same with the second experiment except the parameters a and b. First, we set b = 0.8 and 
0, 1 x 10~ 3 , 1 x 10~ 2 , and 1 x 10 _1 . Second , we set a = 1 x 10 -2 and vary 



vary a as a 

b as b = 0.1, 0.5, 1, and 5. From Fig. 11(a) , we can see that the optimal a is chosen as the 



variance of the small coefficients. Smaller a will result in larger misalignment, and larger 
a will cause slow convergence. From Fig. 11(b), we conclude that 6=1 shows the best 
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Figure 11: The effect of parameters a and b on the proposed algorithm for near sparse 
system. 

performance. Either too large or too small b will result in slower convergence. 

6 Conclusion 

In order to improve the performance of ZA-LMS for near sparse system identification, an 
improved algorithm, DWZA-LMS algorithm, is proposed in this paper by adding a window 
to the zero-point attractor in ZA-LMS algorithm and utilizing the magnitude of estimation 
error to weight the zero-point attractor. Such improvement can adjust the zero-point attrac- 
tion force dynamically to accelerate the convergence rate with no computational complexity 
increased. In addition, the mean square convergence condition, steady-state MSE and pa- 
rameter selection of the proposed algorithm are theoretically analyzed. Finally, computer 
simulations demonstrate the improvement of the proposed algorithm and effectiveness of 
the analysis. 
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